Shuttle Debris Impact Tool Assessment 
Using the Modern Design of Experiments 


Computational tools have been developed to estimate thermal and mechanical reentry 
loads experienced by the Space Shuttle Orbiter as the result of cavities in the Thermal 
Protection System (TPS). Such cavities can be caused by impact from ice or insulating 
foam debris shed from the External Tank (ET) on liftoff. The reentry loads depend on 
cavity geometry and certain Shuttle state variables, among other factors. 

Certain simplifying assumptions have been made in the tool development about the cavity 
geometry variables. For example, the cavities are all modeled as “shoeboxes”, with 
rectangular cross-sections and planar walls. So an actual cavity is typically approximated 
with an idealized cavity described in terms of its length, width, and depth, as well as its 
entry angle, exit angle, and side angles (assumed to be the same for both sides). 

As part of a comprehensive assessment of the uncertainty in reentry loads estimated by the 
debris impact assessment tools, an effort has been initiated to quantify the component of 
the uncertainty that is due to imperfect geometry specifications for the debris impact 
cavities. The approach is to compute predicted loads for a set of geometry factor 
combinations sufficient to develop polynomial approximations to the complex, 
nonparametric underlying computational models. Such polynomial models are continuous 
and feature estimable, continuous derivatives, conditions that facilitate the propagation of 
independent variable errors. 

As an additional benefit, once the polynomial models have been developed, they require 
fewer computational resources to execute than the underlying finite element and 
computational fluid dynamics codes, and can generate reentry loads estimates in 
significantly less time. This provides a practical screening capability, in which a large 
number of debris impact cavities can be quickly classified either as harmless, or subject to 
additional analysis with the more comprehensive underlying computational tools. The 
polynomial models also provide useful insights into the sensitivity of reentry loads to 
various cavity geometry variables, and reveal complex interactions among those variables 
that indicate how the sensitivity of one variable depends on the level of one or more other 
variables. For example, the effect of cavity length on certain reentry loads depends on the 
depth of the cavity. Such interactions are clearly displayed in the polynomial response 
models. 

Five independent cavity geometry variables were selected to define the polynomial 
response models, with the decision to hold exit angle fixed at 75°. This decision was 
predicated on preliminary results that revealed relatively little impact of exit angle changes 
on computed reentry loads, at least over the range of 70° to 80°, which prior experience 
suggests is typical for exit angles. In addition to the five independent geometry variables, 
two Shuttle state variables were identified. The first, boundary layer transition time, 
serves as a surrogate for surface roughness and was defined as a categorical variable with 
two fixed levels, 900 seconds and 1200 seconds, selected to bracket the anticipated range 



of this variable. The second, baseline heating, was likewise established as a two-level 
categorical variable, representing low and moderate baseline heating conditions. 

Even with only five independent numerical variables and two categorical variables, the 
number of computational points required to develop a third-order polynomial model could 
be considerable. For example, at least four levels are required in each numerical variable 
to represent a third-order factor dependence, so a full factorial experiment design 
involving four levels in five factors would require 4 5 =1024 points at each of the four 
combinations of high and low levels of the two categorical variables, or 4096 points per 
computational experiment. The intent was to develop such models for each of the 33 
Shuttle body point zones, so over 135,000 points would be required by such a 
comprehensive design. Even at an optimistic computational rate of 10 minutes per point, 
this would require over 22,000 hours of computer time, a totally unrealistic requirement. 
For this reason a decision was taken to apply Modem Design of Experiments (MDOE) 
methods to the development of the 7-factor computational experiments required in each 
body point zone. MDOE methods were originally introduced at NASA Langley Research 
Center in 1997 to extend conventional “DOE” (design of experiments) methods to 
relatively complex and specialized requirements in high-precision aerospace research, 
notably wind tunnel research. Since then, MDOE methods have been applied to a wide 
range of physical and computational experiments to reduce costs and uncertainty, and to 
provide unique insights into factor interactions. 

This paper describes the development of MDOE computational experiments to generate 
response surface models for factor uncertainty propagation in each of the Shuttle’s 33 
body point zones. Special elements of the design are described, including factor 
constraints that preclude certain physically unrealizable combinations of low entry angle 
with short cavity length at specified depths, and shallow side angles with narrow cavities. 
Other special conditions are also described, including methods to cope with missing data 
in certain body point zones. 

The impact of transformations in both dependent and independent variables is discussed, 
as are certain special techniques to account for the nonlinear nature of factor constraints in 
this problem. Scaling issues are described, which drove decisions about data volume 
requirements. The selection of factor combinations to minimize uncertainty in regression 
coefficients is also described, as are the numerous tests used to validate the response 
models, including the analysis of confirmation points for each body point zone. Lack of fit 
errors quantifying how well the MDOE response models reproduce the underlying 
computational models are documented. 

Finally, representative response models are displayed, illustrating how reentry loads are 
predicted to depend on cavity geometry for various Shuttle state variable combinations, 
and the uncertainty in these load predictions due to uncertainty in the cavity dimensions 
and angles will also be presented. The figures below are examples. 


Figure 1 indicates the degree of agreement between the MDOE polynomial response 



surface and the more complex underlying code for the full range of cavity dimensions 
studied in Shuttle body point zone 1450 (length from 2 in. to 10 in., width from 1.4” to 
4”, and depth from 0.1” to 0.6”, entry angle from 10° to 60°, and sidewall angle from 10° 
to 87°). The standard deviation in residuals on the left is 1.26° F, representing 0.5% of 
the mean structure temperature for all cavities modeled. The figure on the right compares 
structure temperature values estimated with the MDOE polynomial model (predicted) to 
corresponding values computed by the underlying code (“actual”). 

Figure 2 shows a representative response surface on the left, representing structure 
temperatures for a 4 in. wide cavity in body point zone 1450 with a 45° entry angle and 
60° side angles, for nominal baseline heating and a boundary layer transition time of 1200 
seconds. On the right is the corresponding distribution of 95% confidence interval half 
widths representing uncertainty in structure temperature that is attributable to uncertainty 
in cavity geometry variables. The assumed uncertainty in estimating cavity dimensions is 
±l/8 th inch (3a). The assumed uncertainty in estimating wall angles is ±5° (2a). 

Note that the response surfaces clearly reveal general features of the response as well as 
some subtle interaction effects. For example, the structure temperature obviously 
increases with both the length and the depth of the cavity, but cavity length has a greater 
effect for deeper cavities than more shallow cavities. That is, a given increase in length 
will cause a greater increase in structure temperature for deep cavities than shallow 
cavities, which is consistent with intuitive expectations. Likewise, increasing the depth of 
a cavity causes the structure temperature to increase more for a longer cavity than a 
shorter one. 

From the plot on the right in figure 2, the uncertainty in structure temperature attributable 
to uncertainty in cavity geometry appears to be only slightly larger for deep cavities than 
for shallow ones, but the uncertainty is significantly greater for longer cavities than for 
shorter ones. This suggests that rather more care is needed in estimating cavity sizes and 
shapes for longer cavities than shorter ones. 


Representative Figures 
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Figure 1 : Agreement between MDOE structure temperature model and underlying code 
for Shuttle body point zone 1450. Standard deviation in residuals on the left is 1.26° F. 
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Figure 2, Left: Structure temperature on reentry in Shuttle body point zone 1450 for 4 in. 
wide cavity with 45° entry angle and 60° side angles, for nominal baseline heating and a 
boundary layer transition time of 1200 seconds. Right: 95% confidence interval half 
widths for uncertainty in structure temperature due to cavity geometry uncertainty. 



